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ABSTRACT 


A  computer  tool  is  developed  for  the  purpose  of  eliciting 
group  utilities  for  multiple  attributes  of  one  complex  system 
relative  to  a  base  line.  The  procedure  accommodates  multiple 
users  simultaneously  providing  anonymous  feedback  to  each 
user  to  aid  in  the  process  of  assessing  utilities. 

The  procedure  provides  complete  visibility  to  a  manager 
(umpire)  of  changes  to  the  data  base,  so  that  the  process 
can  be  monitored  in  real  time.  The  software  is  written  so 
that  it  is  completely  self -documented  and  user  friendly. 
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I.  INTRODUCTION 


Utility  models  are  often  used  for  the  purpose  of  evalua¬ 
ting  complicated  systems  or  to  select  among  competing 
alternatives.  The  utility  assessment  process  usually 
determines  a  collection  of  system  attributes  which  are  used 
collectively  as  surrogates  for  the  system.  A  decision  maker 
(or  group  of  decision  makers)  is  then  asked  to  evaluate  the 
utility  of  each  attribute  of  the  system.  These  unidimensional 
utilities  are  then  combined  into  a  multiattribute  utility 
measure  of  the  entire  system  using  some  sort  of  rational 
aggregation  procedure  which  depends  on  the  properties  of  the 
unidimensional  utilties. 

Multiple  decision  makers  are  frequently  used  for  determining 
the  unidimensional  utilities  for  the  individual  attributes. 

This  is  because  it  is  rare  to  find  one  person  who  is  expert 
on  all  attributes.  Perhaps  nowhere  is  the  old  adage  "two 
heads  are  better  than  one"  more  true  than  in  utility  assess¬ 
ment.  In  this  thesis  a  computer  tool  is  developed  to  aid 
in  the  process  of  obtaining  the  unidimensional  utilities. 

Background  information  about  the  utility  problem  is 
provided  in  Chapter  II.  The  decision  making  problem  is 
discussed;  multiattribute  utility  is  introduced;  and  the 
group  decision  making  with  feedback  problem  is  examined. 

A  case  is  made  for  the  need  for  an  automated  tool  for 
extracting  group  utilities. 
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Chapter  III  discusses  the  data  and  software  requirements 
for  a  computer  tool  used  to  help  subjects  determine  utili¬ 
ties  for  each  of  several  attributes.  Desirable  interactive 
concepts  of  such  a  tool  are  described.  Hardware  require¬ 
ments  for  the  computer  and  input/output  terminals  are  also 
discussed. 

Chapter  IV  describes  the  various  user  programs  that 
have  been  written  to  interact  with  the  subjects  to  obtain 
utilities.  It  also  describes  the  programs  that  are  avail¬ 
able  to  the  monitor  (umpire)  to  allow  him  to  watch  over  the 
process  and  to  keep  track  of  the  status  of  each  user.  Also 
included  in  Chapter  IV  are  descriptions  of  various  utility 
programs  that  were  written  to  guarantee  data  base  integrity 
and  to  aid  in  the  analysis  of  the  utility  data. 

Chapter  V  provides  a  user's  manual  with  a  sample  terminal 
session  as  an  example  of  the  use  of  the  tool.  The  user's 
manual  should  suffice  for  documentation  to  be  provdided 
to  a  user  as  to  what  he  is  required  to  do  to  utilize  the 
antomated  procedure. 

Finally,  in  Chapter  VI,  we  discuss  present  limitations 
of  the  procedure  in  terms  of  the  number  of  users,  the 
number  of  attributes,  total  core  and  the  like.  We  also 
describe  possible  future  extensions  of  the  process  to  allow 
for  enhanced  graphical  output,  and  we  discuss  other  appli¬ 
cations  of  the  tool  outside  the  area  of  utility  assessment 
that  the  procedure  can  be  used  for  with  only  minor  changes. 
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II.  BACKGROUND 


A.  SUBJECTIVE  EVALUATION  OF  ALTERNATIVES 

Consider  the  problem  of  deciding  among  several  possible 

alternatives  which  we  label  as  A., A-,..., A  .  Each  alterna- 

x  2  n 

tive  has  some  value  or  utility  to  us  which  depends  on  the 
state  of  nature  which  is  outside  our  control.  Let  the 
possible  states  of  nature  be  denoted  by  S^ ,  S^  ,  •  .  .  /  S,^ .  For 
each  pair  (S-,A^)  there  is  a  result  r . ^  (see  figure  1).  The 
collection  of  results  are  what  have  value  or  utility  to  the 
decision  maker(s)  (see  figure  2). 

Decision  theory  is  concerned  with  how  decision  makers 
should  select  among  competing  alternatives  in  such  a  frame¬ 
work.  The  theory  considers  as  separate  cases  decision  making 
under  uncertainty  and  decision  making  under  risk.  In  the 
former,  the  probabilities  for  the  different  states  of  nature 
are  assumed  known;  in  the  latter  the  probabilities  are  unknown. 
In  both  cases,  however,  the  decision  maker (s)  is  required 
to  assess  the  utility  u^  of  each  result  r  ^  .  The  utility 
assignments  are  subjective.  We  assume  that  they  have  been 
made  rationally  in  accordance  with  the  set  of  axioms  of  von 
Neumann  and  Morgens tern  [Ref.  1]  . 

As  an  example  of  this  decision  framework  consider  a 
problem  of  selecting  among  two  available  aircraft  and  one  in 
development  for  the  purpose  of  providing  close  air  support 
for  a  mission  planned  against  enemy  armored  forces.  The 
three  alternatives  are; 


Al  =  A- 4 

A2  =  F~4 
A3  =  A-X 

The  states  of  nature  that  may  affect  the  results  of  the 
mission  are  the  visibility  at  the  time  of  the  mission. 

Let  us  consider  the  states 
=  clear  day 
S2  =  clear  night 
=  cloudy  day 

Now  suppose  that  the  result  matrix  is  given  as  follows 


Note  in  the  table  above  that  the  results  need  not  be 
quantitative.  Finally,  suppose  that  the  decision  maker (s) 
has  assessed  the  following  utilities  for  the  results  shown 
in  the  table : 
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There  are  many  different  procedures  for  selecting  among 
the  alternatives  after  the  alternatives  and  the  states  have 
been  identified  and  the  utilities  assessed.  No  attempt  will 
be  made  to  discuss  the  different  procedures.  For  a  good 
discussion  see  Keeny  and  Raiff  [Ref.  2],  Zimmermann  [Ref.  3] 
Raiffa  [Ref.  4]  and  Fishburn  [Ref.  5].  In  this  thesis  we 
are  concerned  primarily  with  the  problem  of  eliciting  the 
utilities  from  the  decision  maker (s). 

B.  MULTIATTRIBUTE  UTILITY  MODEL 

The  utility  measurement  of  complicated  systems  is  one  of 
the  most  difficult  problems  facing  decision  makers.  The 
measurement  of  utility  must  take  into  account  all  the  antici 
pated  uses  and  conditions  for  which  the  system  would  be 
utilized . 

One  approach  to  assessing  utilities  of  complex  alterna¬ 
tives  is  to  ask  decision  makers  to  assess  the  utility  of  the 
alternatives  directly,  without  explicitly  determining  utili¬ 
ties  of  the  individual  attributes.  This  is  what  most  of  us 
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do  in  our  minds  when  making  everyday  decisions.  The  decision 
makers  may  think  hard  and  seriously  about  the  alternatives 
and  attributes  of  the  alternatives,  but  no  formal  model  is 
developed,  and  no  formal  analysis  is  done.  The  approach  places 
quite  a  burden  on  the  decision  maker  forcing  him  to  integrate 
in  his  mind  many  different  types  of  information  such  as  the 
scenarios  (states  of  nature),  the  attributes,  the  importance 
of  the  attributes  and  tradeoffs.  This  approach  may  lead  to 
good  decisions  but  it  is  generally  unsatisfactory  because  it 
provides  no  'hudit  trail"  for  justifying  its  conclusions  to 
others.  Since  the  model  is  not  specified,  no  scrutiny  can 
be  made  of  the  assumptions  or  factors  that  led  to  the  con¬ 
clusion.  Another  disadvantage  is  that  the  approach  does  not 
allow  for  sensitivity  analysis  or  discovery  of  which  attri¬ 
butes  or  characteristics  of  the  alternative  systems  are  most 
important. 

A  second  approach  is  to  identify  the  key  attributes  of 
the  alternative  systems  that  have  value  to  the  decision 
makers.  Let  represent  r  such  attributes,  and 

let  =  X^ (A)  represent  the  level  of  attribute  X^  possessed 
by  alternative  A.  These  levels  are  used  collectively  as  a 
surrogate  for  the  alternative  systems.  Single  dimensional 
utilities  ui(xl)  are  then  assessed  for  the  levels  of  the 
attributes.  This  approach  thus  allows  a  decision  maker  to 
focus  on  each  attribute  one  at  a  time.  If  care  is  taken  in 
deriving  the  list  of  attributes  so  that  the  set  "captures  the 
essence"  of  the  alternatives,  then  this  approach  minimizes 
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the  likelihood  that  important  considerations  "fall  through 
the  cracks".  It  also  insures  that  the  final  overall  assess 
ments  will  not  be  influenced  unjustifiably  by  placing 
too  much  importance  on  only  a  few  of  the  attributes  of  the 
system.  The  single  dimensional  utilities  are  then  aggre¬ 
gated  into  an  overall  system  utility  as  follows: 


U (A)  =  U (x1,x2, . . . ,xr)  =  f (u1 (x1) , . . . ,u  (x  ) ) 

There  is  quite  a  bit  of  recent  literature  on  the  form  that 
the  aggregating  function  f  should  take.  See  especially 
Keeny  and  Raiffa  [Ref.  2].  One  form  of  f  that  is  commonly 
used  is 


f  (U1(x1) . ur(xr)  ) 


r 

I  W.u. (x. )  . 

i=l  1  1 


This  form  can  be  shown  to  be  valid  when  certain  types  of 
independence  conditions  over  preferences  for  the  attributes 
are  satisfied. 

The  actual  form  of  the  utility  aggregation  function  is 
not  of  concern  in  this  thesis.  Instead,  this  thesis  concen 
trates  on  the  task  of  finding  the  individual  attribute 
utilities  U^(x^). 

C.  GROUP  DECISION  MAKING  WITH  FEEDBACK 

A  single  decision  maker  is  rarely  qualified  to  evaluate 
and  assess  the  utility  of  each  attribute  of  a  complicated 
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system.  One  would  probably  be  better  off  getting  a  group 
of  experts  together  to  share  their  experience  and  assess 
the  utilities  as  a  group.  Getting  a  single  number  out  of 
the  information  supplied  by  a  group  is  a  problem  by  itself. 

In  reference  [6]  a  methodology  for  group  decision  making 
with  feedback  is  proposed. 

The  methodology  suggests  a  procedure  that  has  three 
basic  steps: 

1.  Utility  assessment. 

2.  Information  aggregation. 

3.  Feedback. 

In  the  first  step,  the  group  members  assess  utilities 
for  each  attribute  of  the  system.  The  procedure  administrator 
then  aggregates  all  the  information  about  each  attribute 
and  feeds  information  back  to  the  group  members.  The  group 
members  examine  the  responses  of  their  colleagues  and  modify 
their  own  responses  as  they  desire.  See  figure  3. 

Because  utility  values  are  not  unique  (they  unique  up  to 
a  linear  transformation)  and  because  relative  utilities  and 
values  are  frequently  easier  to  evaluate  than  are  absolute 
values,  we  have  chosen  to  collect  the  required  information 
in  terms  of  value  ratios  with  one  system  selected  as  a 
baseline  for  comparison.  With  the  ratio  scale  (fig.  4)  a 
group  member  is  required  to  provide  two  numbers  as  data  for 
each  attribute. 

1.  The  first  is  the  ratio  of  the  value  of  the  new  system 
to  the  value  of  baseline  system.  For  example:  if  the 
range  of  the  new  system  is  150  NM  and  the  range  of  the  baseline 
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radar  is  100  NM  the  value  ratio  is  1.5.  The  ratio  is  also 
convenient  for  handling  attributes  which  are  non-measurable. 

For  example,  if  a  subject  thinks  the  new  radar  is  30%  more 
reliable  than  the  baseline,  the  reliability  value  ratio  would 
be  1.3 

2.  The  second  is  the  ratio  os  the  utility  of  the  new 
system  to  the  utility  of  the  baseline  system.  For  example: 
if  having  50%  more  range  on  the  new  radar  makes  it  have  a 
utility  twice  as  great  as  the  utility  for  the  baseline,  the 
utility  ratio  for  range  is  2.0. 

Since  a  single  person  is  rarely  an  expert  in  all  of  the 
attributes  of  a  complex  system,  the  group  members  are  asked 
to  provide  a  self -prof iciency  rating  for  each  attribute. 

This  is  done  in  the  first  iteration  of  the  procedure.  After 
all  subjects  have  submitted  an  attribute,  the  data  are 
summarized  and  fed  back  in  graphical  output.  The  graph  for  a 
given  attribute  contains  as  many  points  as  there  are  group 
members.  The  group  members  are  encouraged  to  examine  the 
feedback  and  modify  their  last  inputs  if  they  so  desire. 

Once  the  data  are  in  a  database  on  a  computer,  the  wide 
variety  of  statistical  tools  can  be  used  to  evaluate  and  aggre¬ 
gate  the  final  answers  into  single  utility  numbers  for  all 
attributes,  which  may  then  be  combined  into  an  overall  system 
utility  value. 

Administering  this  procedure  manually  is  very  time  consuming 
and  awkward.  An  iterative  computer  program  is  needed  to 
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handle  the  data  collection  and  feedback  process.  This 
thesis  describes  such  an  interactive  computer  program. 


III.  SOFTWARE  AND  INPUT/QUTPUT  REQUIREMENTS 

A.  DATA  CHARACTERISTICS 

Essentially  two  types  of  data  are  required.  The  first 
type  includes  data  which  are  not  affected  by  feedback  and 
which  will  likely  be  entered  only  once  by  each  user.  This 
includes  the  user  profile  and  information  about  the  user's 
self-proficiency  evaluation  on  each  attribute  and  quadrant 
selection.  The  second  type  is  more  dynamic,  being  directly 
affected  by  feedback  information  on  the  evaluations  of  the 
cohorts  of  the  user.  This  data  consists  of  the  pair  of 
numbers,  value  rates  and  utility  ratio,  for  each  attribute. 

The  user  may  update  these  values  as  often  as  he  desires. 

The  analysis  that  will  be  performed  on  the  input  data 
requires  that  the  entire  data  input  process  be  reconstructed. 
The  analyst  needs  to  know  the  entire  sequence  of  inputs  for 
each  user  for  each  attribute.  Furthermore,  he  needs  to 
trace  through  the  input  chronologically  so  that  he  can  tell 
what  feedback  may  have  influenced  specific  inputs.  This 
requires  that  the  software  insert  a  clock  time  with  each  data 
entry.  Since  the  data  analysis  will  be  performed  on  the  Naval 
Postgraduate  School  IBM  360/67  computer  to  take  advantage  of 
the  powerful  data  analysis  software  already  available  in 
APL,  the  data  will  have  to  be  transportable  in  a  format 
acceptable  to  the  IBM  computer. 


B.  SOFTWARE  CONSIDERATIONS 


Many  considerations  influenced  the  design  of  the  software. 
Most  important  is  the  requirement  that  each  user  simultaneously 
access  the  data  base  interactively.  Also  since  the  users 
will  be  available  for  only  a  short  amount  of  time  ,  extensive 
training  in  the  use  of  the  software  will  not  be  feasible. 

Thus  the  programs  must  be  self -exp lantory  containing  internal 
documentation  and  they  must  be  user  friendly  assuming  no 
prior  computer  training.  The  software  should  be  written  to 
protect  the  identity  of  each  user  from  the  other  users  to 
prevent  intimidation,  but  should  allow  the  project  adminis¬ 
trator  to  monitor  the  performance  of  each  user  in  real  time. 
This  is  to  allow  the  administrator  to  observe  the  progress 
of  each  user.  The  administrator  can  thus  detect  problems  that 
the  users  may  be  having  and  communicate  instructions  to 
selected  users  during  a  session.  Finally,  to  be  useful  as 
a  general  tool  for  the  assessment  of  group  utilities  of 
several  attributes  of  one  system  relative  to  a  specified 
baseline,  the  programs  should  be  flexible  to  allow  for 
changes  in  the  systems  being  evaluated  without  major  software 
changes . 

C .  INTERACTIVE  CONCEPTS 

It  is  difficult  to  provide  a  friendly  interface  between 
a  sophisticated  unforgiving  machine  and  a  user  assumed  to  be 
untrained  in  the  use  of  computers.  The  burden  of  accomplishing 
the  interface  falls  on  the  software.  Thus  the  software  must 
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incorporate  many  human  factors  considerations.  The  informa¬ 
tion  displayed  to  the  user  should  be  simple,  clear  and  con- 

* 

cise  so  that  the  user  can  grasp  the  essential  information 
quickly.  The  time  delay  from  keyboard  entry  to  the  terminal 

4. 

response  should  be  no  more  than  a  few  seconds.  The  software 
should  provide  the  user  an  option  with  regard  to  the  amount 
of  detail  contained  in  the  instructions  printed  at  the  user's 
terminal.  As  users  become  more  experienced  during  a  terminal 
session,  he  will  likely  grow  weary  of  repeatedly  reading  the 
same  detailed  instructions.  He  should  therefore  be  allowed 
to  reduce  the  verbosity  of  instructions.  In  addition  to 
allowing  the  usef  to  reduce  verbosity,  the  software  should 
also  allow  the  user  to  request  additional  information  or 
instructions.  A  "help"  feature  should  be  built  into  the  soft- 
ware  to  allow  the  user  to  obtain  online  documentation  at  any 
time  during  a  session  without  disturbing  the  flow  of  the 
program.  Finally,  the  software  should  be  built  to  provide 
protection  of  the  data  base  from  either  the  malicious  intent 
of  a  user  or  from  accidental  or  erroneous  responses. 

D.  INPUT/OUTPUT  TERMINALS 

Because  of  costs,  portability,  and  response  requirements 
a  "dumb"  video  CRT  terminal  with  a  1200  baud-rate  transmission 
capability,  standard  alphanumeric  keyboard  entry,  23  lines 
per  screen,  and  80  characters  per  line  was  selected  as  the 
basis  input/output  device  around  which  the  software  would 
be  designed.  Many  different  terminals  satisfy  the  above 
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requirements  providing  ready  availability  for  testing  and 
implementation  at  the  Naval  Postgraduate  School.  This  choice 
of  terminals  allows  the  user  to  view  only  a  single  quadrant 
(which  he  selects)  containing  the  value  ratios  and  the  utility 
ratios  for  a  selected  attribute  since  the  entire  graph  cannot 
be  displayed  with  adequate  resolution.  Fortunately,  this 
does  not  present  serious  problems  since  one  would  expect 
most  or  all  users  to  input  data  into  the  same  quadrant  for  a 
given  attribute.  In  order  to  provide  each  user  access  all 
the  values  for  a  given  attribute  the  software  should  notify 
each  user  of  the  distribution  of  data  over  the  four  quadrants 
and  allow  each  user  to  display  any  selected  quadrant. 

Other  more  expensive  types  of  intelligent  terminals 
could  certainly  enhance  the  application  of  the  process. 

Color  and  graphics  capability  would  allow  the  users  to  dis¬ 
tinguish  among  the  data  according  to  the  self -prof iciencv 
rating  of  the  respondents  and  would  allow  greater  resolution. 
Also  a  split  screen  capability  would  allow  for  some  information, 
such  as  the  attribute  list  to  be  maintained  on  a  portion  of 
the  screen  while  other  parts  are  updated  as  necessary.  This 
would  reduce  the  input/output  response  time  while  providing 
more  informative  displays.  Even  though  the  software  is 
designed  around  the  "dumb"  video  terminal,  it  should  be 
flexible  enough  to  allow  for  easy  modification  for  a  more 
sophisticated  terminal. 
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IV .  SOFTWARE  DESCRIPTION 

A.  THE  USER'S  PROGRAM 

1 .  External  Declarations 

Most  of  the  variables  are  declared  externally  so 
that  they  can  be  used  globally. 

The  maximum  number  of  users  and  attributes  and  the 
si2e  of  the  basic  data  record  is  defined  so  as  to  make  it 
easy  to  change  the  system  capacity. 

The  basic  structures  that  constitutes  the  data  bank 
are  defined  and  character  arrays  are  initialized. 

2 .  Main  (  ) 

The  main  program  handles  the  general  flow  of  the 
process.  It  performs  the  linking,  opening  and  unlinking  of 
the  various  files  needed. 

A  unique  user  number  is  associated  with  each  user 
and  the  basic  loop  of  the  program  is  entered. 

3 .  Intro  (  ) 

The  introduction  routine  explains  the  "rules  of  the 
game"  and  describes  the  basic  procedure  that  the  user  will 
follow. 

Intro  (  )  gives  examples  of  use  and  sample  displays. 
The  user  is  allowed  to  page  back  and  forth  through  the  intro¬ 
ductory  instructions  until  he  feels  that  he  understands  what 
he  is  required  to  do  and  what  options  are  available  to  him. 
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The  user  can  re-enter  the  Intro  (  )  subroutine  at 
almost  any  time  he  wants  by  typing  "I"  (see  subroutine 
attrib  (  ) ) . 

4 .  Prep  (  ) 

The  preparation  routine  performs  the  first  iteration 
and  asks  the  user  for  two  basic  pieces  of  information 
regarding  each  attribute: 

(1)  The  user's  self-rated  proficiency  with  respect 
to  an  attribute  on  a  scale  of  1  (low)  to  5  (high) . 

(2)  The  quadrant  in  which  he  will  enter  utility 
information.  Once  the  user  enters  his  proficiency  it  may 
not  be  changed  at  a  later  time.  However  the  quadrants  may 
be  updated  later  at  any  time. 

After  the  entry  of  the  proficiency  and  the  quadrant  informa¬ 
tion,  the  chosen  quadrant  is  displayed  for  the  given  attri¬ 
bute  and  the  user  is  prompted  to  enter  his  first  evaluation 
of  the  value  and  utility  ratios.  This  sequence  is  repeated 
for  all  the  attributes. 

5 .  Menu  (  ) 

This  menu  routine  displays  to  the  user  the  list  of 
attributes  and  a  number  by  which  the  attribute  will  be 
referenced.  In  each  attribute  there  will  also  be  a  column 
called  "changes",  which  gives  each  user  a  count  of  the 
number  of  times  that  other  users  have  updated  the  information 
for  each  attribute,  since  he  last  made  a  utility  entry  for 
the  attribute. 
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6.  Attrib  (  ) 

This  routine  is  the  main  working  routine.  It 
starts  by  displaying  to  the  user  the  current  utility  data 
for  the  selected  attribute  and  then  prompts  each  user  for 
a  response.  The  user  has  7  available  options  which  he  can 
type. 

"y"  -  indicates  that  the  user  wants  to  enter  new 

utility  data.  The  user  is  then  prompted  for 
the  value  and  utility  ratios. 

"n"  -  indicates  that  no  new  data  is  to  be  entered. 

The  program  returns  to  main  (  ) ,  to  prompt 
for  a  new  attribute. 

"a"  -  indicates  that  the  user  wants  to  look  at  the 

attribute  list.  The  menu  (  )  is  displayed. 

"q"  -  indicates  that  he  wants  a  display  of  the 

utility  data  for  a  different  quadrant. 

"I"  -  calls  up  the  introduction  routine. 

"E"  -  indicates  that  the  user  wishes  to  terminate 

the  session. 

"h"  -  or  any  other  character  not  within  this  list 

of  legal  options.  A  list  of  the  options  is 
displayed. 

7 .  Qutdata  (  ) 

The  outdata  routine  generates  a  CRT  display  of  the 
most  recent  data  entered  by  all  the  users  in  any  selected 
attribute.  The  data  is  displayed  to  each  user  in  the 
quadrant  which  he  selects. 
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A  fresh  copy  of  the  data  is  read  from  the  file 
system  and  the  attribute  value  ratio  and  the  utility  ratio 
are  extracted.  The  display  quadrant  is  determined  and 
the  proper  display  subroutines  are  called. 

3  *  Qi /  ^2*  Q3 '  ^4 

Four  quadrant  subroutines,  one  for  each  quadrant 
translate  the  attribute  value  and  the  utility  ratios  into 
the  proper  scale  for  display.  The  routines  will  prepare  a 
character  array  to  be  displayed.  If  more  than  one  point 
of  data  falls  in  one  cell,  the  number  of  points  is  displayed 
as  a  digit.  Otherwise,  the  blank  is  displayed. 

9 .  Graph  1  (  )  -  Graph  4  (  ) 

Four  graph  subroutines,  one  per  quadrant,  display  the 
proper  axes  and  the  data  points  in  that  quadrant.  Each  sub¬ 
routine  also  displays  a  small  four-quadrant  figure  showing 
the  number-  of  current  entries  in  each  quadrant  for  the 
selected  attribute. 

10 .  Indata  (  ) 

The  indata  subroutine  creates  the  new  data  records 
and  writes  them  into  the  proper  files  in  the  file  system. 
Before  writing,  the  subroutine  will  look  for  permission  to 
write  into  the  data  base.  As  soon  as  permission  is  given, 
the  information  is  written  into  the  files. 


B.  THE  MONITOR'S  PROGRAMS 


1.  Clear 

The  program  "clear"  initializes  the  data  base  files. 

A  warning  is  given  to  the  monitor  that  the  program  will 
destroy  any  existing  data  in  the  file.  The  monitor  can 
quit  and  save  the  file  contents  elsewhere. 

2.  Atlst 

The  program  "atlst"  enables  the  monitor  to  create 
or  update  an  attribute  list  at  any  time. 

3 .  Tbox 

The  program  "tbox"  is  a  small  data  base  administrator, 
created  to  maintain  data  base  integrity.  It  prevents  the 
user's  processes  from  overwriting  information  in  the  data 
bank.  The  program  will  accept  requests  for  write  permission 
into  the  data  files  from  all  the  users  and  will  issue  the 
write  permission  to  only  one  user  at  a  time. 

4 .  An 

The  program  "an"  enables  the  monitor  to  monitor  the 
progress  of  the  working  session. 

The  program  permits  the  monitor  to  do  the  following: 

a.  Data  records  in  numerical  presentation  can  be 
displayed  for  any  single  user  or  all  users  and  any  single 
attribute  or  all  attributes. 

b.  The  data  in  graphical  presentation  (as  the  user 
sees  it) ,  can  be  displayed  for  any  user  and  any  attribute, 
as  they  are  at  any  selected  time,  present  or  past. 


c.  The  data  in  graphical  presentation  (as  the  user 
sees  it),  can  be  displayed  for  any  user  and  any  attribute, 
as  they  are  at  any  selected  time,  present  or  past  -  but  only 
for  users  from  a  chosen  proficiency  and  up. 

The  program  runs  continuously  until  terminated  by 

the  monitor. 

5.  Boxstop 

This  little  program  will  send  a  message  to  the 
administrator  tbox  to  stop  execution  orderly. 

6.  LI 

The  program  " 11"  takes  an  entire  data  base  and  trans¬ 
forms  all  the  integers  into  long  integers,  so  they  will  be 
compatible  with  the  IBM/360  format. 

7 .  Tape 

The  command  file  "tape"  loads  the  output  of  the 
program  "11"  on  a  tape  for  transportation  to  other  computers 
for  analysis  of  the  data. 

8 .  Tapdsk,  Extpdsk 

The  FORTRAN  program  "tapdsk"  reads  the  data  from  a 

t 

tape  and  stores  it  into  CMS  files. 

The  program  "EXTPDSK"  is  an  EXEC  routine  under  the 
cp/cms  time  sharing  system  running  on  the  IBM  360/67  at 
NPGS .  The  program  defines  the  proper  CMS  files  for  storage 
of  the  data  bank  taken  from  the  PDP-11  and  runs  the  program 
"tapdsk". 
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VI.  LIMITATIONS  AND  EXTENSIONS 


The  current  implementation  of  the  group  decision  making 
procedure  on  the  PDP  11/50  reserves  half  the  available  data 
core  storage  for  the  user's  input  data  records.  Some  extra 
core  can  still  be  allocated  for  that  purpose  but  not  a  signifi¬ 
cant  amount. 

The  current  limit  on  the  total  number  of  records  is  2K. 

Any  number  of  users,  attributes,  and  inputs  per  user  per 
attribute  whose  product  does  not  exceed  2K  is  acceptable. 

In  order  to  be  able  to  handle  a  significantly  greater  amount 
of  data  one  could  page  the  data  bank. 

In  the  current  application  the  graphical  display  used  is 
a  "dumb"  CRT  terminal.  The  procedure  would  be  enhanced  by 
using  a  split-screen  terminal  or  a  full  graphic  display 
terminal.  Only  minor  changes  to  the  main  user  program  "tt.c" 
(in  the  graphical  display  subroutines)  would  be  required  to 
adapt  the  application  to  a  more  sophisticated  terminal. 

The  procedure  can  be  used  to  evaluate  more  than  one 
alternative  and  any  number  of  scenarios  by  saving  the  data 
bank  files  under  different  names  and  reinitializing  the 
procedure  for  each  new  alternative-scenario  combination. 

The  limited  data  base  management  and  analysis  system 
described  in  this  thesis  for  the  group  decision  making  pro¬ 
cedure  can  easily  be  adapted  for  several  other  applications. 

The  basic  features  that  the  process  provides  are: 
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1.  Multiple  users  simultaneously  assessing  a  common 
dynamic  data  base. 

2.  Immediate  selected  feedback  with  anonymity  provided 
to  the  users. 

3.  Protection  of  the  data  base  so  that  one  user's  inputs 
are  never  allowed  to  overwrite  another's  inputs. 

4.  Complete  visibility  of  the  actions  that  take  place 

to  a  monitor  or  umpire  who  could  communicate  with  any 
user . 

5.  Users  allowed  to  proceed  at  their  own  pace. 

6.  Complete  internal  self-documentation  with  system 
prompts  requesting  all  necessary  data. 

7.  A  capability  of  reconstructing  the  entire  process  over 
time  of  each  user. 

These  features  are  useful  for  various  types  of  information 
gathering  or  decision  making  tasks.  One  especially  important 
application  is  to  the  area  of  interactive  war  gaming. 
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FIGURE  5 
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APPENDIX  A 


SOFTWARE  DESIGN  AND  IMPLEMENTATION 

A.  THE  DATA  STRUCTURE 

I.  Conceptual  View 

Two  major  types  of  data  are  invovled:  static  and 
dynamic.  The  static  data  includes  a  fixed  amount  of  informa¬ 
tion  for  each  user-attribute  pair.  The  dynamic  data  consists 
of  the  record  built  from  each  value  and  attribute  ratio 
entered  by  the  users. 

Figure  5  depicts  the  static  data  body  S,  the  dynamic 
data  body  D  and  their  respective  data  records  Rg  and  R^. 

There  are  (*  of  users)  x  (#  of  attributes)  records  Rg.  The 
contents  of  the  information  in  Rg  may  change  but  the  locations 
of  the  fields  are  fixed. 

In  the  body  D  the  top  horizontal  plane  contains  the  last 
information  entered  about  an  attribute-user  pair  in  records 
of  the  form  R^.  When  a  user  updates  the  values  for  an 
attribute,  the  new  data  is  placed  on  the  top  of  the  stack 
that  grows  down  from  the  horizontal  plane.  At  any  given 
time  the  user  program  has  access  only  to  the  data  records 
that  forms  the  top  horizontal  plane  (the  top  of  the  stacks) . 

The  program  monitor  needs  access  to  the  entire  data  base 
accumulated  during  a  session.  For  every  user-attribute  pair 
he  needs  to  be  able  to  see  the  self-rated  proficiency,  and 
the  quadrant  for  display,  then  all  the  value  and  utility 
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ratios  that  were  entered  and  the  time  of  entry.  Later,  after 
the  data  are  gathered,  the  analyst  will  need  to  reconstruct 
the  data  input  history  in  order  to  be  able  to  investigate 
the  input  pattern  of  all  the  users  and  thus  be  able  to  detect 
any  biases  if  they  exist. 

2 .  The  Implementation 

The  "C"  language  was  selected  because  of  its  pointer 
handling  facilities,  its  ability  to  easily  define  and  operate 
on  complex  data  structures,  and  its  simple  handling  of 
character  information. 

A  data  record  consists  of  the  following  fields: 


user  number 
attribute  number  j 
attribute  value  ratio 
utility  ratio 
time  1  1 


the  indexing  pair 


the  basic  user's  input 


system  clock  time  at  data  entry 


time  2  J 

pointer  —  a  link  to  the  previous  record  entered  by 
the  user  for  an  attribute 


For  each  indexing  pair  (user  number;  attribute  number) 

a  record  R  will  contain  the  fields: 
s 

pointer  —  a  link  to  the  corresponding  stack-top  in  D 

counter  —  the  number  of  changes  made  to  this  attribute 
since  this  user  last  updated  it 
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proficiency  —  the  self-rated  proficiency  of  the  user 
with  respect  to  this  attribute 

quadrant  —  the  quadrant  in  which  the  next  display 
will  be  presented. 


The  structure: 


struct  attr{ 

int  user [NUSERS] , count [NUSERS ] ,prof [NUSERS] , quadr [NUSERS ] 


contains  all  the  R  records  for  a  certain  attribute.  S 

s 

consists  of  R  such  structures  where  k  is  the  number  of 
attributes . 

The  structure : 

struct  data { 

int  dm,  pd;  usn,  atrn,  usl,  vsl,  time  [2]; 

} 

contains  a  data  record  R^.  An  array  of  2K  such  records  is 
defined  and  contains  the  data  bank;  it  is  large  enough  to 
allow  each  of  20  users  to  input  4  different  pairs  for  each 
of  25  attributes.  This  should  be  sufficiently  large  to 
accomodate  most  applications.  Since  individual  users  will 
differ  in  updating  patterns,  no  attempt  is  made  to  allocate 
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a  fixed  number  of  blocks  to  each  user.  Instead,  records 
from  the  bank  are  allocated  sequentially. 

The  structures  that  contain  S  are  stored  and  main¬ 
tained  in  the  file  "atb"  and  the  ones  that  form  D  in  the 
file  "bnk".  Both  will  be  referred  to  $  the  data  bank. 

Searching  and  updating  the  data  bank  on  the  files 
is  a  very  time  consuming  I/O  operation,  hence,  periodically 
the  files  are  read  into  the  appropriate  structures  in  core 
and  the  searching  is  performed  in  core. 

Note:  Since  the  stacks  in  D  are  maintained  as 

singly  linked  lists,  no  updates  are  needed  in  the  previous 
records  when  a  record  is  added  on  the  top  of  a  stack.  The 
pointer  field  in  the  new  record  will  point  to  the  old  top, 
and  the  only  update  is  done  in  the  pointer  field  of  the 
proper  record  in  the  structure  "attr"  that  now  will  point 
to  the  new  top. 

B .  DATA  INTEGRITY 

Since  UNIX  is  a  time  sharing  system  all  communications 
and  all  data  shared  by  two  or  more  users  must  take  place 
through  use  of  the  file  system.  At  any  time  only  one  user 
has  access  to  the  CPU.  Care  must  be  taken  that  the  data  in 
the  files  that  are  shared  by  multiple  users  is  protected  so 
that  the  input  from  one  user  cannot  write  over  another '  s 
data. 
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The  data  which  we  collect  are  stored  in  two  files. 


The  static  data  base  (contained  in  the  structures  "att") 
are  stored  in  the  file  called  "atb”.  This  file  has  a  fixed 
size  and  each  field  has  a  fixed  predetermined  location. 
Therefore  two  different  users  will  always  write  into  different 
locations . 

Protection  of  this  data  file  is  therefore  no  problem. 

A  simple  algorithm  in  the  user's  program  takes  care  to  write 
all  information  in  the  "atb"  file  in  the  proper  locations. 

The  dynamic  portion  of  the  data  base  is  contained  in  the 
file  "bnk" .  Space  is  not  preallocated  in  this  file,  nor  is 
specific  information  stored  in  predetermined  locations. 

New  information  is  appended  to  the  data  base  without  erasing 
any  old  data.  The  software  must  keep  track  of  the  size  of 
the  file  and  the  location  of  the  next  record  to  be  added 
to  the  data  base.  At  any  instant  two  or  more  users  may 
simultaneously  decide  to  add  new  data  to  the  file  and  the 
same  location  may  be  given  to  multiple  users.  Therefore 
the  input  from  one  user  could  write  over  the  input  of  another 
user,  and  the  former  value  would  be  lost  forever.  Therefore 
a  protection  mechanism  was  developed  to  make  sure  that  the 
problem  never  occurs. 

C.  THE  TICKET  BOX  SOLUTION 

A  sequencing  approach  described  in  Ref.  [11]  was  adapted 
to  the  problem  at  hand  to  protect  the  data  base  from  one  user 
writing  over  the  data  of  another  user.  The  approach  is 
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analogous  to  the  method  used  increasingly  by  many  businesses 
to  accommodate  a  multi-server,  single  queueing  process.  A 
customer  arriving  at  a  store  desiring  service  is  assigned  a 
numbered  ticket.  In  front  of  the  waiting  customers  is  a 
display  showing  the  number  of  the  customer  presently  receiving 
service.  When  a  service  is  completed  the  display  counter 
is  incremented  by  one  and  the  waiting  customer  holding  the 
matching  ticket  is  serviced. 

The  same  principle  is  used  to  provide  the  data  base 
protection  in  our  problem.  It  is  handled  with  an  administra¬ 
tive  program  called  "tbox".  It  runs  concurrently  with  the 
users'  programs  receiving  write  requests  from  the  users, 
issuing  sequential  tickets,  incrementing  the  counter  when 
a  write  is  completed  and  checking  for  matches  between  tickets 
and  the  counter.  Its  operation  is  described  by  the  flowchart 
in  Figure  6. 

The  files  called  "REQUEST"  and  "TICKET"  contain  a  fixed 
number  of  fields  equal  to  the  number  of  users.  The  file 
"COUNT"  contains  a  single  integer. 

Each  user  who  wants  to  write  new  information  into  the 
data  base  first  requests  permission  to  write  through  the 
file  "REQUEST".  A  check  is  then  made  of  the  "TICKET"  file 
to  determine  if  the  administrative  program  "tbox"  has  issued 
a  ticket  to  the  user.  This  check  is  repeated  until  a  ticket 
has  been  issued.  Once  a  ticket  is  issued  a  companion  of  the 
ticket  number  is  made  with  the  "COUNT"  file  to  determine  if 
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the  user  has  write  permission.  When  the  write  operation  is 
completed  the  user's  program  sends  a  message  to  the  admini¬ 
strator  to  increment  the  counter  so  new  write  permission  can 
be  granted.  The  programs  are  sent  to  "sleep  (  ) "  when  a 
check  fails  since  there  is  no  sense  in  wasting  the  time 
in  checking;  no  other  program  that  can  change  the  status 
runs  at  the  same  time  . 
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Ihis  pronrain  will  link  and  execute  the  Group  Decision  Making 
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reA'J(f<1brl«lf)SH<0 


I 


► 


nnnnn 


extpdsk  exec 


BLKSIZE  32 

LRECL  32  BLKSIZE  32 

LRECL  32  BLKSIZE  32 


FILEDEF  2  TAP 2  RECFM  F  LRECL  32 
FILEDEF  3  DSK  ATTR  FILE  RECFM  F 
FILEDEF  4  DSK  BANK  FILE  RECFM  F 
LOAD  TAP DSK  (XEQ) 


tapdsk  fortran 


THIS  PROGRAM  WILL  READ  TWO  FILES  FROM  A  TAPE  AND 
WRITES  THEM  INTO  A  DISK  ON  THE  CP/CMS  FILE  SYSTEM 


INTEGER* 4  A (8) 

1  =  0 

1  READ (2,100, END=500)  A 
100  FORMAT  ( 8A4 ) 

WRITE (3, 200)  A 
200  FORMAT (8A4) 

1  =  1  +  1 
GO  TO  1 

500  END  FILE  2 

WRITE (6, 505)  I 

505  FORMAT ('  END  OF  FILE  ATTR.', 18,'  RECORDS.') 
1  =  0 

2  READ (2,100, END=  600)  A 
WRITE (4, 200)  A 

1  =  1  +  1 
GO  TO  2 

600  END  FILE  2 

WRITE (6, 605)  I 

605  FORMAT ('  END  OF  FILE  BANK. ',18,'  RECORDS.’) 
STOP 
END 
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